Public object storage, such as Amazon Web Services (AWS) Simple Storage Service (S3) and Google Cloud Storage, is an inexpensive and popular way to store large amount of data. Thus, the trend in the IT industry is to move the bulk of analytics and application data to such object storages, which are provided by third parties as services. At the same time, users of these object storage services typically want to continue accessing and analyzing much of that data using applications that run on compute clusters that may reside on either public clouds or on-premise computing environments.
However, accessing data from the public object storages can be very expensive. Many enterprise customers already have on-premise data centers, which have large amounts of idle computing power. However, the process to copy data from a public object storage to on-premise data centers for processing is slow and expensive because network bandwidth is expensive and network latency is high. Another common problem of accessing object data is the interface. AWS S3 provides a REST API via HTTPS, but many existing applications are built on top of Portable Operating System Interface (POSIX) file system interface.
Throughout the description, similar reference numbers may be used to identify similar elements.